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Abstract 

We argue that the spectral theory of non-reversible Markov chains may often be more 
effectively cast within the framework of the naturally associated weighted-Loo space L^, 
instead of the usual Hilbert space L2 = L2{tt), where tt is the invariant measure of the 
chain. This observation is, in part, based on the following results. A discrete-time Markov 
chain with values in a general state space is geometrically ergodic if and only if its transition 
kernel admits a spectral gap in LY^. If the chain is reversible, the same equivalence holds 
with L2 in place of L^, but in the absence of reversibility it fails: There are (necessarily 
non-reversible, geometrically ergodic) chains that admit a spectral gap in but not in 
1/2- Moreover, if a chain admits a spectral gap in L2, then for any h € L2 there exists a 
Lyapunov function V;, G Li such that Vh dominates h and the chain admits a spectral gap 
in . The relationship between the size of the spectral gap in or L2, and the rate at 
which the chain converges to equilibrium is also briefly discussed. 
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1 Introduction and Main Results 

There is increasing interest in spectral theory and rates of convergence for Markov chains. 
Research is motivated by elegant mathematics as well as a range of applications. In particular, 
one of the most effective general methodologies used to establish bounds on the convergence 
rate of a geometrically ergodic chain is via an analysis of the spectrum of the chain's transition 
kernel. See, e.g., [16, 22, 21, 8, 5, 12, 11, 13, 23, 19, 3, 2, 17, 14, 7, 6], and the relevant references 
therein. 

The word spectrum naturally invites techniques grounded in a Hilbert space framework. 
The majority of quantitative results on rates of convergence are obtained using such methods, 
within the Hilbert space L2 = L2(vr), where vr denotes the stationary distribution of the 
Markov chain in question. Indeed, most successful studies have been carried out for Markov 
chains that are reversible, in which case a key to analysis is the fact that the transition kernel, 
viewed as a linear operator on L2, is self-adjoint. In this paper we argue that, in the absence 
of reversibility, the Hilbert space framework may not be the appropriate setting for spectral 
analysis. 

To be specific, let X = {X(n) : n > 0} denote a discrete-time Markov chain with values 
on a general state space X. We assume that X is equipped with a countably generated sigma- 
algebra B. The distribution of X is described by its initial state X{0) = xq £ X and the 
transition semigroup {P" : n > 0}, where, for each n, 

A) := Fv{X{n) G A \ X{0) = x}, xeX, AeB. 

For simplicity we write P for the one-step kernel P^. Recall that each P", like any (not 
necessarily probabilistic) kernel Q{x,dy) acts on functions F : X ^ C and signed measures v 
on (X, B), via, 

QF{-)= [ Q{;dy)F{y) and uQ{-) = [ u{dx)Q{x,-), 

whenever the integrals exist. Throughout the paper, we assume that the chain X = {X{n)} 
is ip -irreducible and aperiodic; cf. [15, 18]. This means that there is a cr-finite measure '0 on 
(X, B) such that, for any A e B with V'(^) > 0, and any x e X, 

P^{x,A) > 0, for all n sufficiently large. 

Moreover, we assume that is maximal in the sense that any other such -0' is absolutely 
continuous with respect to tp. 

1.1 Geometric ergodicity 

The natural class of chains to consider in the present context is that of geometrically ergodic 
chains, namely, chains with the property that there exists an invariant measure vr on (X, B) 
and functions /3 : X ^ (0, 1) and C : X — > [1, 00), such that, 

•) - ttIItv < C{x)p{x)"-, for all n > 0, vr-a.e. x gX, 

where ||/x||tv := sup^gg denotes the total variation norm on signed measures. Under ip- 

irreducibility and aperiodicity this is equivalent [15, 19] to the seemingly stronger requirement 
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that there is a single constant p £ (0,1), a constant B < oo and a vr-a.e. finite function 
y : X ^ [1, oo], such that, 

||P"(x, •) - 7r||y < B y(x)p", for all n > 0, vr-a.e. x e X, (1) 

where ||^||y := sup{| f F dfi\ : F £ L^} denotes the F-norm on signed measures, and where 
denotes the weighted-Loo space consisting of all measurable functions F: X ^ C with, 

F y:=sup^-ff<oo. 2) 

Another equivalent and operationally simpler definition of geometric ergodicity for a ip- 
irreducible, aperiodic chain X = {X{n)}, is that it satisfies the following drift criterion [15]: 



There is a function V : X — > [1, oo], a small set C C X, 
and constants 5 > 0, b < oo, such that: 

PV <{1- 6)V + blc . 



(V4) 



We then say that the chain is geometrically ergodic with Lyapunov function V. In (V4) it 
is always assumed that the Lyapunov function V is finite for at least one x (and then it is 
necessarily finite ip-a.e.). Also, recall that a set C £ B is small if there exist n > 1, e > and 
a probability measure v on (X,B) such that, P'^{x,A) > eIc{x)i'{A), for all x £ X, A £ B. 

Our first result relates geometric ergodicity to the spectral properties of the kernel P. Its 
proof, given at the end of Section 3, is based on ideas from [12]. See Section 2 for more precise 
definitions. 

Proposition 1.1. A ^lJ -irreducible and aperiodic Markov chain X = {X{n)} is geometrically 
ergodic with Lyapunov function V if and only if P admits a spectral gap in . 



1.2 Reversibility 

Recall that the chain X = {X(n)} is called reversible if there is a probability measure vr on 
iX,B) satisfying the detailed balance conditions, 

Tr{dx)P{x,dy) = 'n{dy)P{y,dx). 

This is equivalent to saying that the linear operator P is self-adjoint on the space L2 = L2{tt) 
of (measurable) functions F : X ^ C that are square-integrable under tt, endowed with the 
inner product {F, G) = J FG* dir, where '*' denotes the complex conjugate operation. 

The following result is the natural analog of Proposition 1.1 for reversible chains. Its proof, 
given in Section 3, is partly based on results in [19]. 

Proposition 1.2. A reversible, tjj -irreducible and aperiodic Markov chain X = {X{n)} is 
geometrically ergodic if and only if P admits a spectral gap in L2. 
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1.3 Spectral theory 

The main question addressed in this paper is whether the reversibihty assumption of Propo- 
sition 1.2 can be relaxed. In other words, whether the space L2 can be used to characterize 
geometric ergodicity hke was in Proposition 1.1. One direction is true without reversibil- 
ity: A spectral gap in L2 implies that the chain is "geometrically ergodic in L2" [19] [20], and 
this implies the existence of a Lyapunov function V satisfying (V4) [20]. Therefore, the chain 
is geometrically ergodic in the sense of [12], where it is also shown that it must admit a central 
gap in L^. A direct, explicit construction of a Lyapunov function is given in our first main 
result stated next, where quantitative information about Vh is also obtained. It is proved in 
Section 3. 

Theorem 1.3. Suppose that a if: -irreducible, aperiodic chain X = {X(n)} admits a spectral 
gap in L2. Then, for any h G L2, there is n-integrable function Vh, such that the chain is 
geometrically ergodic with Lyapunov function Vh and h G . 

But the other direction may not hold in the absence of reversibility. Based on earlier 
counterexamples constructed by Haggstrom [9, 10] and Bradley [1], in Section 3 we prove the 
following: 

Theorem 1.4. There exists a ip -irreducible, aperiodic Markov chain X = {A'(n)} which is 
geometrically ergodic but does not admit a spectral gap in L2. 

1.4 Convergence rates 

The existence of a spectral gap is intimately connected to the exponential convergence rate 
for a ^-irreducible, aperiodic Markov chain. For example, if the chain is reversible, we have 
the following well-known, quantitative bound. See Section 2 for detailed definitions; the result 
follows from the results in [19], combined with Lemma 2.2 given in Section 2. 

Proposition 1.5. Suppose that a reversible chain X = {X(n)} is ip -irreducible, aperiodic, 
and has initial distribution fi. If the chain X admits a spectral gap 82 > in L2, then, 



where the L2-norm on signed measures v is defined as the L2{'iT)-norm of the density di^/dn if 
it exists, and is set equal to infinity otherwise. 

In the absence of reversibility, the size of the spectral gap in precisely determines the 
exponential convergence rate of any geometrically ergodic chain. The result of the following 
proposition is stated in Lemma 2.3 in Section 2. 

Proposition 1.6. Suppose that the chain X = {X{n)} is ip -irreducible and aperiodic. If it 
admits a spectral gap 6v > in L^, then, for vr-a.e. x, 




n 



n > 1 




tt\\v = log(l - dv). 



In fact, the convergence is uniform in that: 




(3) 
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Section 2 contains precise definitions regarding the spectrum and the spectral gap of the 
kernel P acting either on L2 or the weighted-Loo space L^. Simple properties of the spectrum 
are also stated and proved. Section 3 contains the proofs of the first four results stated above. 



2 Spectra and Geometric Ergodicity 

We begin by giving precise definitions for the spectrum and spectral gap of the transition 
kernel P, viewed as a linear operator. The spectrum depends on the domain of P, for which 
we consider two possibilities: 

(i) The Hilbert space L2 = L2{tt), equipped with the norm ||-F||2 = [/ dvr]"'^/^. 

(ii) The Banach space L^, with norm || • ||y defined in (2). 

In either case, the spectrum is defined as the set of nonzero A G C for which the inverse 
(/A — P)~^ does not exist as a bounded linear operator on the domain of P. The transition 
kernel admits a spectral gap if there exists eo > such that S f] {z : \z\ > 1 — eq} is finite, and 
contains only poles of finite multiplicity; see [12, Section 4] for more details. The spectrum 
is denoted ^2 when P is viewed as a linear operator on L2, and it is denoted Sy when P is 
viewed as a linear operator on L^. 

The induced operator norm of a linear operator P : — > is defined as usual via, 

III Pill ll^^llv 
l\P\lv •=s^P-|iTqi — ' 

ll-f" \\v 

where the supremum is over all F G satisfying ||-F||y 7^ 0. An analogous definition gives 
the induced operator norm |||-P|||2 of a linear operator P acting on L2. 

For a '(/'-irreducible, aperiodic chain X = {X{n)}, geometric ergodicity expressed in the 
form (1) implies that converges to a rank-one operator, at a geometric rate: For some 
constants -B < 00, p G (0, 1), 

IP'' - 1 vrlll^ < P p" , n > 0, (4) 

where the outer product 1 ® vr denotes the kernel 1 ® 7r{x, dy) = n^dy). It follows that the 
inverse [/A — P + 1 ® 7r]~^ exists as a bounded linear operator on L^, whenever X > p. This 
in turn implies that P has a single isolated pole at A = 1 in the set {A G C : A > />}, so that 
P admits a spectral gap. 

In Lemma 2.1 we clarify the location of poles when the chain admits a spectral gap in L2 
or L^. 

Lemma 2.1. // a ip -irreducible, aperiodic Markov chain admits a spectral gap in or L2, 
then the only pole on the unit circle in C is X = 1, and this pole has multiplicity one. 

Proof. We present the proof for ; the proof in L2 is identical. 

We first note that the existence of a spectral gap implies ergodicity: There is a left eigen- 
measure /i corresponding to the eigenvalue 1, satisfying fiP = fi and = WpWv < 00. On 

letting 7r( • ) = |^(- )|/|p(X)| we conclude that vr is super-invariant: ttP > vr. Since '/r(X) = 1 
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we must have invariance. The ergodic theorem for positive recurrent Markov chains imphes 
that E[G{X{n)) \ X{0) = x] ^ J G dn, as n — > oo, whenever G £ Li^n). 

Ergodicity rules out the existence of multiple eigenfunctions corresponding to A = 1. Hence, 
if this pole has multiplicity greater than one, then there is a generalized eigenfunction h G 
satisfying, 

Ph = h + l. 

Iterating gives P'^h{x) = E[h{X{n))\ X{{)) = x] = h{x) + n for n > 1. This rules out 
ergodicity, and proves that A = 1 has multiplicity one. 

We now show that if A G Sy with |A| = 1, then A = 1. To see this, let h G denote an 
eigenfunction, Ph = Xh. Iterating, we obtain, 

E[h{X{n)) I X{0) =x] = /i(x)A". 

Then, letting n — > oo, the right-hand-side converges to J hdn for a.e. x., so that A = 1 and 
h{x) = J hdn, TT-a.e. □ 



Therefore, for a -i/^-irreducible, aperiodic chain, the existence of a spectral gap in L2 is 
equivalent to the existence of a single eigenvalue A = 1 on the unit circle, which has multiplicity 
one. The spectral gap 62 is then defined as, 

62 = 1- sup{|A| : A G 52, A / 1}, 

and similarly for 6v- 

Next we state two well-known, alternative expressions for the L2-spectral gap 82 of a 
reversible chain. See, e.g., [19, Theorem 2.1] and [4, Proposition VIII. 1.11]. 

Lemma 2.2. Suppose X is a ip -irreducible, aperiodic, reversible Markov chain. Then, its 
L2- spectral gap 82 admits the alternative characterizations, 



82 = 1 — sup|— ^ — --^ : signed measures V with viX) = ||i^||2 / o| 

/ N 1/n 

= 1- lim (|||P"- l07r||| 

n— >oo V 



where the limit is the usual spectral radius of the semigroup {-P"} generated by the kernel 
P = P — 1 (S> IT, acting on functions in L2{tt). 

A similar result holds for 8v, even in the absence of reversibility; see, e.g., [13]. 

Lemma 2.3. Suppose X is a ^p -irreducible, aperiodic Markov chain. Then, its L^-spectral 
gap 8v admits the following alternative characterization in terms of the spectral radius, 



1- lim IIP" 



5V 

n— »oo 



1/n 
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3 Proofs 

First we prove Theorem 1.3. The following notation will be useful throughout this section. 

For a Markov chain X = {X{n)}, the first hitting time and first return time to a set C £ B 
are defined, respectively, by, 

ac ■■= mm{n > : X{n) £ C}; 
Tc :=min{n > 1 : X{n) G C}. 

Conditional on X(0) = x, the expectation operator corresponding to the measure defining 
the distribution of the process X = {X{n)} is denoted Ex-(-), so that, for example, P^F{x) = 
E[F{X{n)) I X{Q) = x] = Ex[F{X{n))]. For an arbitrary signed measure /x on {X,B), we write 
/i(-F) for J F dfi, for any function F -.X ^ C for which the integral exists. 

Proof of Theorem 1.3. Since TT{h?) < oo, and the chain is ^/^-irreducible, it follows that there 
exists an increasing sequence of h'^ -regular sets providing a vr-a.e. covering of X [15, Theorem 
14.2.5]. That is, there is a sequence of sets {Sr : r £ Z+j such that 7r(5r) ^ 1 as r — > oo, 
Sr C Sr+i for each r, and the following bounds hold, 

< OO, for vr-a.e. x 



Vr{x) :=E, [Y^h\X{n)) 



n=Q 

sup Vr{x) < OO. 

xeSr 

Since the chain admits a spectral gap in L2, combining Theorem 2.1 of [19] with Lemma 2.2 
and the results of [20], we have that it is geometrically ergodic. Hence, from [15, Theo- 
rem 15.4.2] it follows that there exists a sequence of Kendall sets providing a vr-a.e. covering 
of X. That is, there is a sequence of sets {Kr : r £ Z+} and positive constants {6r : r £ 
satisfying vr(i('r) — > 1 as r ^ 00, Kr C K^+i for each r, and the following bounds hold, 

Ur{x) := Ex[e^p{OrTKr)] < 00, for vr-a.e. x 
sup Ur{x) < 00. 

We also define another collection of sets, 

Cr,m := {x G X : Ur{x) + Vr{x) < m} . 

For each r > 1, these sets are non-decreasing in m, and vr(Cr,m) — > 1 as m — > 00. Moreoever, 
whenever Cr^m £ B~^, this set is both an /i^-regular set and a Kendall set. This follows by 
combining Theorems 14.2.1 and 15.2.1 of [15]. Fix ro and mo so that T^{Cro^mo) > 0. We 
henceforth denote Cro,mo by C, and let > denote a value satisfying the bound, 

Ej, [exp(0rc)] < 00, for vr-a.e. x, 



where the expectation is uniformly bounded over the Kendall set C. 
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The candidate Lyapunov function can now be defined as, 



14(x) := [^(1 + |M^(n))|) exp(ien) 

n=0 



(6) 



We first obtain a bound on this function. Writing, 



Vhix) = E,. exp(i0n)] + ^ E,. [\h{X{n))\ exp(i^n)l{n < ac}] , 

n=0 n.=0 

we see that the first term is finite 7r-a.e. by construction. The square of the second term is 
bounded above, using the Cauchy-Shwartz inequahty, by, 

OO OO CTC 

E^[Y.\h{X{n))\H{n < ac}]E,[j^exp(0n)l{n < ac}] = C/ro(2;)E, exp(0t) 



n=0 



n=0 



t=0 



SO that Vh is finite vr-a.e., and we also easily see that \h\ < Vh so that h G L^. 

Next we show that Vh satisfies (V4): First apply e^^P to the function Vh to obtain, 

TC 

e^'PVh (x) = [^(1 + \hiX{n + 1))|) exp(i0(t + 1)) 

n=l 



(7) 



We have tc = ctq when X(0) G C^. This gives, 

e^'PVh{x) = Vh{x) - (1 + X e 

If X{0) = X € C, then the previous arguments imply that the right-hand-side of (7) is 
finite, and in fact uniformly bounded over x £ C. Combining these results, we conclude that 
there exists a constant 6o such that, 

PVh < e-i^Vh + Mc 

Regular sets are necessarily small [15, Theorem 11.3.11] so that this is a version of the drift 
inequality (V4). 

Finally note that, by the fact that (V4) implies the weaker drift condition (V3) of [15], the 
function Vh is vr-integrable by [15, Theorem 14.0.1]. □ 

Theorem 1.3 states that (V4) holds for a Lyapounov function Vh with h G L^. If this 
could be strengthened to show that for every geometrically ergodic chain and any h € L2, the 
chain was geometrically ergodic with a Lyapunov function Vh that had /i^ G L^, then the 
central limit theorem would hold for the partial sums of h{X(n)) [15, Theorem 17.0.1]. But 
this is not generally possible: 

Proposition 3.1. There exists a geometrically ergodic Markov chain on a countable state 
space X and a function G £ L2 with mean 7t{G) = 0, for which the central limit theorem fails 
in that the normalized partial sums, 



^ n— 1 



n > 1, 



(8) 



converge neither to a normal distribution nor to a point mass. 
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The result of the proposition appears in [9, Theorem 1.3], and an earher counterexample in 
[1] yields the same conclusion. Based on these counterexamples we now show that geometric 
ergodicity does not imply a spectral gap in the Hilbert space setting. 

Proof of Theorem 1.4- Suppose that the Markov chain X = {X{n)} constructed in Propo- 
sition 3.1 does admit a spectral gap in L2. Then its autocorrelation fuction decays geomet- 
rically fast, for any h £ L2: Assuming without loss of generality that '/r(/i) = 0, and letting 
Rh{n) = 'ir(hP'^h), for all n, we have the bound, 



\R{n)\ < v^7r(/i2)7r((P"/i)2), n > 1. 

Applying Theorem 1.3, we conclude that the right-hand-side decays geometrically fast as 
n — > 00. Consequently, the sequence of normalized sums, 

n-l 



i=0 

is uniformly bounded in L2, i.e.. 



1 " " 

S^:= — Y,h{X{i^), n>l, 



limsupE^[52] < J2 



00 

n=—oo 



where denotes the expectation operator corresponing to the stationary version of the 
chain. However, this is impossible for the choice of the function /i = G as in Proposition 3.1: 
In [9, p. 81] it is shown that the corresponding normalized sums in (8) fail to define a tight 
sequence of probability distributions. This is a consequence of [9, Lemma 3.2]. 

This contradiction establishes the claim that the Markov chain of Proposition 3.1 cannot 
admit a spectral gap in L2. □ 

Finally we prove Propositions 1.1 and 1.2. 

Proof of Proposition 1.1. The equivalence stated in the proposition is obtained on combining 
Lemma 2.1 with [12, Proposition 4.6]. To explain this, we introduce new terminology: The 
transition kernel is called V -uniform if A = 1 is the only pole on the unit circle in C, and this 
pole has multiplicity one. Proposition 4.6 of [12] states that geometric ergodicity with resepct 
to a Lyapunov function V is equivalent to ^/-uniformity of the kernel P. Consequently, the 
direct part of the proposition holds, since ^/-uniformity of P implies that it admits a spectral 
gap in L^. 

Conversely, if the chain admits a spectral gap in L^, then Lemma 2.1 states that P 
is ^/-uniform. Applying Proposition 4.6 of [12] once more, we conclude that the chain is 
geometrically ergodic with the same Lyapunov function V . □ 

Proof of Proposition 1.2. The forward direction of the statement of the proposition is 
contained in [19] and [20]. 

The converse again follows from Lemma 2.1 and a minor modification of the arguments 
used in [12, Proposition 4.6]. If the chain admits a spectral gap in L2, then the lemma states 
that A = 1 has multiplicity one, and that this is the only pole on the unit circle in C. It follows 
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that for some p < I, the inverse [zl — {P — 1 '^)]~^ exists as a bounded hnear operator on 
L2, whenever \z\ > p. Denote bp = sup|||[2;/ — (P — 1 (g) 7r)]~-^|||2 : \z\ = p}, where ||| • is the 
induced operator norm on L2. 

Following the proof of [12, Theorem 4.1], we conclude that finiteness of bp implies a form 
of geometric ergodicity: For any g £ L2, 




Therefore, the L2-norm of the left-hand-side is bounded by 6p||(7||2. This gives, 

||P"5-vr(5)||2<6p||5||2/>"+\ n>l. 
It follows from [15, Theorem 15.4.3] that the Markov chain is geometrically ergodic. □ 
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